Real-time sparse-sampled Ptychographic imaging through deep neural networks

Ptychography has rapidly grown in the fields of X-ray and electron imaging for its unprecedented ability to achieve nano or atomic scale resolution while simultaneously retrieving chemical or magnetic information from a sample. A ptychographic reconstruction is achieved by means of solving a complex inverse problem that imposes constraints both on the acquisition and on the analysis of the data, which typically precludes real-time imaging due to computational cost involved in solving this inverse problem. In this work we propose PtychoNN, a novel approach to solve the ptychography reconstruction problem based on deep convolutional neural networks. We demonstrate how the proposed method can be used to predict real-space structure and phase at each scan point solely from the corresponding far-field diffraction data. The presented results demonstrate how PtychoNN can effectively be used on experimental data, being able to generate high quality reconstructions of a sample up to hundreds of times faster than state-of-the-art ptychography reconstruction solutions once trained. By surpassing the typical constraints of iterative model-based methods, we can significantly relax the data acquisition sampling conditions and produce equally satisfactory reconstructions. Besides drastically accelerating acquisition and analysis, this capability can enable new imaging scenarios that were not possible before, in cases of dose sensitive, dynamic and extremely voluminous samples

Multislice forward modeling of Coherent Surface Scattering Imaging on surface and interfacial structures

To study nanostructures on substrates, surface-sensitive reflection-geometry scattering techniques such as grazing incident small angle x-ray scattering are commonly used to yield an averaged statistical structural information of the surface sample. Grazing incidence geometry can probe the absolute three-dimensional structural morphology of the sample if a highly coherent beam is used. Coherent Surface Scattering Imaging (CSSI) is a powerful yet non-invasive technique similar to Coherent X-ray Diffractive Imaging (CDI) but performed at small angles and grazing-incidence reflection geometry. A challenge with CSSI is that conventional CDI reconstruction techniques cannot be directly applied to CSSI because the Fourier-transform-based forward models cannot reproduce the dynamical scattering phenomenon near the critical angle of total external reflection of the substrate-supported samples. To overcome this challenge, we have developed a multislice forward model which can successfully simulate the dynamical or multi-beam scattering generated from surface structures and the underlying substrate. The forward model is also demonstrated to be able to reconstruct an elongated 3D pattern from a single shot scattering image in the CSSI geometry through fast-performing CUDA-assisted PyTorch optimization with automatic differentiation.Comment: 12 pages, 4 figures, 1 tabl

AI-assisted Automated Workflow for Real-time X-ray Ptychography Data Analysis via Federated Resources

We present an end-to-end automated workflow that uses large-scale remote compute resources and an embedded GPU platform at the edge to enable AI/ML-accelerated real-time analysis of data collected for x-ray ptychography. Ptychography is a lensless method that is being used to image samples through a simultaneous numerical inversion of a large number of diffraction patterns from adjacent overlapping scan positions. This acquisition method can enable nanoscale imaging with x-rays and electrons, but this often requires very large experimental datasets and commensurately high turnaround times, which can limit experimental capabilities such as real-time experimental steering and low-latency monitoring. In this work, we introduce a software system that can automate ptychography data analysis tasks. We accelerate the data analysis pipeline by using a modified version of PtychoNN -- an ML-based approach to solve phase retrieval problem that shows two orders of magnitude speedup compared to traditional iterative methods. Further, our system coordinates and overlaps different data analysis tasks to minimize synchronization overhead between different stages of the workflow. We evaluate our workflow system with real-world experimental workloads from the 26ID beamline at Advanced Photon Source and ThetaGPU cluster at Argonne Leadership Computing Resources.Comment: 7 pages, 1 figure, to be published in High Performance Computing for Imaging Conference, Electronic Imaging (HPCI 2023

Elucidation of Relaxation Dynamics Beyond Equilibrium Through AI-informed X-ray Photon Correlation Spectroscopy

Understanding and interpreting dynamics of functional materials \textit{in situ} is a grand challenge in physics and materials science due to the difficulty of experimentally probing materials at varied length and time scales. X-ray photon correlation spectroscopy (XPCS) is uniquely well-suited for characterizing materials dynamics over wide-ranging time scales, however spatial and temporal heterogeneity in material behavior can make interpretation of experimental XPCS data difficult. In this work we have developed an unsupervised deep learning (DL) framework for automated classification and interpretation of relaxation dynamics from experimental data without requiring any prior physical knowledge of the system behavior. We demonstrate how this method can be used to rapidly explore large datasets to identify samples of interest, and we apply this approach to directly correlate bulk properties of a model system to microscopic dynamics. Importantly, this DL framework is material and process agnostic, marking a concrete step towards autonomous materials discovery

Deep learning at the edge enables real-time streaming ptychographic imaging

Coherent microscopy techniques provide an unparalleled multi-scale view of materials across scientific and technological fields, from structural materials to quantum devices, from integrated circuits to biological cells. Driven by the construction of brighter sources and high-rate detectors, coherent X-ray microscopy methods like ptychography are poised to revolutionize nanoscale materials characterization. However, associated significant increases in data and compute needs mean that conventional approaches no longer suffice for recovering sample images in real-time from high-speed coherent imaging experiments. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz. The proposed AI-enabled workflow eliminates the sampling constraints imposed by traditional ptychography, allowing low dose imaging using orders of magnitude less data than required by traditional methods